{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3c10e1d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.Intertek与隆基绿能联手,为开拓新市场加强PSI+COC认证检测领域合作\n",
      "隆基绿能该条新闻舆情评分为4\n",
      "https://www.cet.com.cn/wzsy/cyzx/3401220.shtml\n",
      "2.隆基绿能钟宝申:预计2023年电池组件出货量85GW,硅片出货量130GW\n",
      "隆基绿能该条新闻舆情评分为4\n",
      "https://finance.eastmoney.com/a/202306292765190669.html\n",
      "3.隆基绿能\n",
      "隆基绿能该条新闻舆情评分为-10\n",
      "http://stock.hexun.com/2023-06-29/209101122.html\n",
      "4.隆基绿能:受高温天气影响 预计全球潜在光伏需求有望持续释放\n",
      "隆基绿能该条新闻舆情评分为-5\n",
      "http://news.10jqka.com.cn/20230629/c648412354.shtml\n",
      "5.【调研快报】隆基绿能接待阳光人寿等多家机构调研\n",
      "隆基绿能该条新闻舆情评分为4\n",
      "https://finance.eastmoney.com/a/202306292765144834.html\n",
      "6.隆基绿能:将继续开展多种电池技术研究 不押注单一路线\n",
      "隆基绿能该条新闻舆情评分为-10\n",
      "https://emwap.eastmoney.com/a/202306292765124858.html\n",
      "7.隆基绿能(601012.SH):公司一般根据订单安排生产,尺寸切换较为灵活\n",
      "隆基绿能该条新闻舆情评分为-5\n",
      "http://sc.stock.cnfol.com/ggzixun/20230629/30257132.shtml\n",
      "8.隆基绿能(601012.SH):鄂尔多斯30GWN型TOPCon电池产能于5月底开始...\n",
      "隆基绿能该条新闻舆情评分为0\n",
      "http://sc.stock.cnfol.com/ggzixun/20230629/30257130.shtml\n",
      "9.隆基绿能:二季度公司硅片和组件出货量环比不断增加\n",
      "隆基绿能该条新闻舆情评分为4\n",
      "https://baijiahao.baidu.com/s?id=1770007299927174318&amp;wfr=spider&amp;for=pc\n",
      "10.隆基绿能(601012.SH):将继续开展多种电池技术的研究\n",
      "隆基绿能该条新闻舆情评分为-5\n",
      "https://baijiahao.baidu.com/s?id=1770006796414549661&amp;wfr=spider&amp;for=pc\n",
      "——————————————————————————————\n",
      "隆基绿能该公司百度新闻爬取成功\n",
      "1.亿晶光电6月29日快速上涨\n",
      "亿晶光电该条新闻舆情评分为4\n",
      "https://finance.eastmoney.com/a/202306292764956584.html\n",
      "2.亿晶光电:6月28日获融资买入8402.41万元,占当日流入资金比例18.25%\n",
      "亿晶光电该条新闻舆情评分为-2\n",
      "http://stock.10jqka.com.cn/20230629/c648403787.shtml\n",
      "3.亿晶光电:融资净买入3458.84万元,融资余额4.16亿元(06-28)\n",
      "亿晶光电该条新闻舆情评分为-6\n",
      "https://finance.eastmoney.com/a/202306292764282823.html\n",
      "4.亿晶光电:公司股东情况请详见定期报告\n",
      "亿晶光电该条新闻舆情评分为4\n",
      "http://app.myzaker.com/news/article.php?pk=649bee51b15ec061220cb708\n",
      "5.亿晶光电(600537):该股换手率大于8%(06-28)-和讯网\n",
      "亿晶光电该条新闻舆情评分为0\n",
      "http://stock.hexun.com/2023-06-28/209086581.html\n",
      "6.亿晶光电(600537)6月27日主力资金净买入9891.56万元\n",
      "亿晶光电该条新闻舆情评分为-6\n",
      "https://stock.stockstar.com/RB2023062800007041.shtml\n",
      "7.亿晶光电:融资净偿还590.44万元,融资余额3.81亿元(06-27)\n",
      "亿晶光电该条新闻舆情评分为-6\n",
      "https://finance.eastmoney.com/a/202306282762726859.html\n",
      "8.亿晶光电:底部启动!业绩暴涨146%,归母净利同比翻倍!投资潜力巨大...\n",
      "亿晶光电该条新闻舆情评分为12\n",
      "http://stock.hexun.com/2023-06-27/209076925.html\n",
      "9.鑫元清洁能源混合发起式A重仓股亿晶光电涨7.11%\n",
      "亿晶光电该条新闻舆情评分为0\n",
      "http://fund.10jqka.com.cn/20230627/c648334596.shtml\n",
      "10.华耀光电加码垂直一体化布局 冲刺创业板IPO\n",
      "亿晶光电该条新闻舆情评分为0\n",
      "https://finance.eastmoney.com/a/202306272761269872.html\n",
      "——————————————————————————————\n",
      "亿晶光电该公司百度新闻爬取成功\n",
      "1.钧达股份6月29日快速反弹\n",
      "钧达股份该条新闻舆情评分为0\n",
      "https://finance.eastmoney.com/a/202306292764923125.html\n",
      "2.钧达股份(002865):6月29日技术指标出现观望信号-“黑三兵”-和讯网\n",
      "钧达股份该条新闻舆情评分为-5\n",
      "http://stock.hexun.com/2023-06-29/209095310.html\n",
      "3.钧达股份06月28日被深股通减持24.05万股\n",
      "钧达股份该条新闻舆情评分为-5\n",
      "https://finance.eastmoney.com/a/202306292764609443.html\n",
      "4.股票行情快报:钧达股份(002865)6月28日主力资金净卖出3741.71万元\n",
      "钧达股份该条新闻舆情评分为-1\n",
      "https://stock.stockstar.com/RB2023062800029322.shtml\n",
      "5.钧达股份: 关于2022年股票期权激励计划首次授予部分第一个行权期...\n",
      "钧达股份该条新闻舆情评分为4\n",
      "https://stock.stockstar.com/notice/SN2023062700025904.shtml\n",
      "6.6月 26 日钧达股份(002865)龙虎榜数据:机构净卖出 9178.65 万元...\n",
      "钧达股份该条新闻舆情评分为-1\n",
      "http://www.myzaker.com/article/6499578cb15ec001c22e65a8\n",
      "7.钧达股份龙虎榜数据(6月26日)\n",
      "钧达股份该条新闻舆情评分为-1\n",
      "https://finance.eastmoney.com/a/202306262761054361.html\n",
      "8.钧达股份\n",
      "钧达股份该条新闻舆情评分为0\n",
      "http://www.stcn.com/quotes/index/sz002865.html\n",
      "9.钧达股份(002865):持股5%以上股东被动稀释股份变动超过1%\n",
      "钧达股份该条新闻舆情评分为-5\n",
      "https://www.cfi.net.cn/p20230621003540.html\n",
      "10.钧达股份:董事长陆小红拟减持不超157.2838万股公司股份\n",
      "钧达股份该条新闻舆情评分为-5\n",
      "https://baijiahao.baidu.com/s?id=1769310111305290225&amp;wfr=spider&amp;for=pc\n",
      "——————————————————————————————\n",
      "钧达股份该公司百度新闻爬取成功\n",
      "1.固德威06月28日被沪股通减持13.03万股\n",
      "固德威该条新闻舆情评分为-5\n",
      "https://finance.eastmoney.com/a/202306292764647616.html\n",
      "2.固德威:6月28日获融资买入4731.02万元,占当日流入资金比例15.69%\n",
      "固德威该条新闻舆情评分为-2\n",
      "http://stock.10jqka.com.cn/20230629/c648403889.shtml\n",
      "3.沪深股通|固德威6月28日获外资卖出0.12%股份\n",
      "固德威该条新闻舆情评分为-5\n",
      "http://yuanchuang.10jqka.com.cn/20230629/c648401408.shtml\n",
      "4.股票行情快报:固德威(688390)6月28日主力资金净买入4608.28万元\n",
      "固德威该条新闻舆情评分为4\n",
      "https://stock.stockstar.com/RB2023062800028276.shtml\n",
      "5.固德威技术股份有限公司 2023年第二次临时股东大会决议公告\n",
      "固德威该条新闻舆情评分为-5\n",
      "http://m.epaper.zqrb.cn/html/2023-06/28/content_959233.htm\n",
      "6.固德威(688390):2023年第二次临时股东大会决议\n",
      "固德威该条新闻舆情评分为-5\n",
      "https://www.cfi.net.cn/p20230627003040.html\n",
      "7.固德威: 2023年第二次临时股东大会决议公告\n",
      "固德威该条新闻舆情评分为-5\n",
      "https://stock.stockstar.com/notice/SN2023062700027519.shtml\n",
      "8.固德威06月21日被沪股通减持52.11万股\n",
      "固德威该条新闻舆情评分为-5\n",
      "https://finance.eastmoney.com/a/202306222759568142.html\n",
      "9.固德威:融资净偿还858.79万元,融资余额6亿元(06-19)\n",
      "固德威该条新闻舆情评分为-6\n",
      "https://finance.eastmoney.com/a/202306202756690828.html\n",
      "10.固德威\n",
      "固德威该条新闻舆情评分为0\n",
      "http://www.stcn.com/quotes/index/sh688390.html\n",
      "——————————————————————————————\n",
      "固德威该公司百度新闻爬取成功\n"
     ]
    }
   ],
   "source": [
    "# =============================================================================\n",
    "# 5.3 舆情评分系统搭建 by 王宇韬 代码更新：www.huaxiaozhi.com 资料下载区\n",
    "# =============================================================================\n",
    "\n",
    "import requests  # 首先导入requests库和正则表达式re库\n",
    "import re\n",
    "\n",
    "headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36'}\n",
    "\n",
    "\n",
    "'''舆情评分系统 - 通过新闻标题和新闻正文来进行评分'''\n",
    "def baidu(company):\n",
    "    url = 'http://www.baidu.com/s?tn=news&rtt=4&bsst=1&cl=2&wd=' + company  # 其中设置rtt=4则为按时间排序，如果rtt=1则为按焦点排序\n",
    "    res = requests.get(url, headers=headers).text\n",
    "    #print(res)\n",
    "\n",
    "    # 正则表达式编写，这边为了代码简洁，只演示了标题和链接（正则更新）\n",
    "    p_href = '<h3 class=\"news-title_1YtI1 \"><a href=\"(.*?)\"'\n",
    "    href = re.findall(p_href, res, re.S)\n",
    "    p_title = '<h3 class=\"news-title_1YtI1 \">.*?>(.*?)</a>'\n",
    "    title = re.findall(p_title, res, re.S)\n",
    "\n",
    "    # 舆情评分版本4\n",
    "    score = []\n",
    "    keywords = ['高温', '电力短缺', '订单', '监督', '单一路线', '研究', '乏力','减持','卖出','黑三兵','股份变动','偿还','决议']  # 这个关键词列表可以自己定义，这里只是为了演示\n",
    "    keywords11 = ['联手' ,'激励计划', '订单增加', '需求释放', '出货量', '买入', '迅猛','业绩暴涨','快速上涨','净利同比翻倍','流入资金','大规模稳定供应']  # 这个关键词列表可以自己定义，这里只是为了演示\n",
    "    for i in range(len(title)):\n",
    "        num = 0\n",
    "        # 获取新闻正文\n",
    "        try:\n",
    "            article = requests.get(href[i], headers=headers, timeout=10).text\n",
    "\n",
    "        except:\n",
    "            article = '爬取失败'\n",
    "        # 解决可能存在的乱码问题\n",
    "        try:\n",
    "            article = article.encode('ISO-8859-1').decode('utf-8')  # 方法3\n",
    "        except:\n",
    "            try:\n",
    "                article = article.encode('ISO-8859-1').decode('gbk')  # 方法2\n",
    "            except:\n",
    "                article = article  # 方法1\n",
    "        # 只筛选真正的正文内容，旁边的滚动新闻之类的内容忽略\n",
    "        \n",
    "        p_article = '<p>(.*?)</p>'  # 有的时候p标签里还有class等无关内容，所以更严谨的写法是<p.*?>(.*?)</p>\n",
    "        article_main = re.findall(p_article, article)  # 获取<p>标签里的正文信息\n",
    "        article = ''.join(article_main)  # 将列表转换成为字符串\n",
    "        for k in keywords:\n",
    "            if (k in article) or (k in title[i]):\n",
    "                num -= 5\n",
    "        for k in keywords11:\n",
    "            if (k in article) or (k in title[i]):\n",
    "                num += 4\n",
    "        score.append(num)\n",
    "\n",
    "    for i in range(len(title)):\n",
    "        title[i] = title[i].strip()  # strip()函数用来取消字符串两端的换行或者空格，不过目前（2020-10）并没有换行或空格，所以其实不写这一行也没事\n",
    "        title[i] = re.sub('<.*?>', '', title[i])  # 核心，用re.sub()函数来替换不重要的内容\n",
    "        print(str(i + 1) + '.' + title[i])\n",
    "        print(company + '该条新闻舆情评分为' + str(score[i]))  # 这边注意，不要写score[i]，因为它是数字，需要通过str()函数进行字符串拼接\n",
    "        print(href[i])\n",
    "    print('——————————————————————————————')  # 这个是当分隔符使用\n",
    "    file1=open('舆情报告.csv','a')\n",
    "    file1.write(\"sfs\"+\"\\n\"+\"\\n\")\n",
    "    for i in range(len(title)):\n",
    "        file1.write(str(i+1)+\".\"+title[i]+\"\\n\")\n",
    "        file1.write(company + '该条新闻舆情评分为' + str(score[i])+\"\\n\")\n",
    "        file1.write(href[i]+\"\\n\")\n",
    "    file1.write(\"__________________________________\"+\"\\n\")\n",
    "    file1.close()\n",
    "companys = ['隆基绿能','亿晶光电','钧达股份','固德威']\n",
    "for i in companys:\n",
    "    try:\n",
    "        baidu(i)\n",
    "        print(i + '该公司百度新闻爬取成功')\n",
    "    except:\n",
    "        print(i + '该公司百度新闻爬取失败')\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8baa39ca",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ce6a37b9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "相关系数r值为-0.05589594912122893，显著性水平P值为0.8781089182942309\n"
     ]
    }
   ],
   "source": [
    "# =============================================================================\n",
    "# 7.3.2 相关系数分析 by 王宇韬&肖金鑫  代码更新：www.huaxiaozhi.com 资料下载区\n",
    "# =============================================================================\n",
    "\n",
    "import pandas as pd\n",
    "from scipy.stats import pearsonr\n",
    "\n",
    "# 读取数据\n",
    "data = pd.read_excel('111.xlsx')\n",
    "# 相关性分析\n",
    "corr = pearsonr(data['score'], data['price'])\n",
    "print('相关系数r值为' + str(corr[0]) + '，显著性水平P值为' + str(corr[1]))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6b6bb30b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>score</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2023-06-29</td>\n",
       "      <td>4</td>\n",
       "      <td>28.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2023-06-19</td>\n",
       "      <td>4</td>\n",
       "      <td>28.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2023-06-26</td>\n",
       "      <td>4</td>\n",
       "      <td>28.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2023-06-29</td>\n",
       "      <td>-10</td>\n",
       "      <td>28.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2023-06-29</td>\n",
       "      <td>-5</td>\n",
       "      <td>28.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2023-06-15</td>\n",
       "      <td>4</td>\n",
       "      <td>28.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2023-06-29</td>\n",
       "      <td>-10</td>\n",
       "      <td>28.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2023-06-29</td>\n",
       "      <td>-5</td>\n",
       "      <td>28.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2023-06-28</td>\n",
       "      <td>0</td>\n",
       "      <td>29.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2023-06-28</td>\n",
       "      <td>4</td>\n",
       "      <td>29.32</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        date  score  price\n",
       "0 2023-06-29      4  28.85\n",
       "1 2023-06-19      4  28.98\n",
       "2 2023-06-26      4  28.01\n",
       "3 2023-06-29    -10  28.85\n",
       "4 2023-06-29     -5  28.85\n",
       "5 2023-06-15      4  28.78\n",
       "6 2023-06-29    -10  28.85\n",
       "7 2023-06-29     -5  28.85\n",
       "8 2023-06-28      0  29.32\n",
       "9 2023-06-28      4  29.32"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50f23683",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
